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Abstract 

In much of the world, the issue of accountability and measurement of educational outcomes is highly controversial. Exit testing is part of 
the movement to ascertain what students have learned and hold institutions and teachers to account. However, compared to the large 
number of teacher washback studies, learner washback research is lacking (Cheng, 2008). By exploring standardized exit test washback in 
Taiwan, this study aims to test Alderson and Wall’s hypotheses (1993) regarding learner washback variability, namely that a test will 
influence 1) degree/depth of learning, 2) attitudes toward methods of learning, and 3) some learners but not others. This study is one of 
very few washback studies that compare learner behaviors before and after the introduction of exit tests, thereby obtaining a clearer picture 
of washback effects on learners. Questionnaires and scores on two exit tests (GEPT and TOEIC) were collected from 589 Taiwanese 
university students before and after the introduction of English exit tests in Taiwan. This study highlights the differential effects of exit 
requirements for different groups of learners depending on their years of study, proficiency levels, and viewpoints of tests. 
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Introduction 

In much of the world, the issue of accountability and measurement of educational outcomes is highly controversial. Exit testing is one part 
of the movement to ascertain what students have learned and to hold institutions and teachers to account (Jacob, 2001; O’Loughlin & 
Arkoudis, 2009; Humphreys and Mousavi, 2010). However, compared to the large number of teacher washback studies, learner washback 
research is lacking (Cheng 2008; Spratt, 2005; Wall, 2000; Watanabe, 2004). By exploring standardized exit test washback, this study 
tested Alderson and Wall’s hypotheses (1993) regarding learner washback variability. In particular, this study aimed to investigate how exit 
tests influence learners’ motivation, learning strategies, and test performance. 


Theoretical Washback Framework 

“Washback” (Alderson & Wall, 1993; Buck, 1988) or “backwash” (Biggs, 1995, 1996; Hughes, 2003) is often defined as the influence that 
tests exert on teaching and learning. To better understand both the scope and mechanism of washback, researchers have proposed various 
models, hypotheses, and concepts. 

Hughes’s (1993) trichotomy of backwash model describes test effects in terms of “participants,” “process (es),” and “product(s).” A test 
could affect “participants” such as teachers, students, administrators, materials writers, and publishers, “all of whose perceptions and 
attitudes toward their work may be affected” (p. 2). “Process” refers to any actions undertaken by participants to complete teaching and 
learning tasks, such as materials development, syllabus design, changes in teaching methods or content, and learning and/or test-taking 










strategies (p. 2). “Product” refers to “learning outcomes (e.g., skills) and the quality of learning (fluency)” (p. 2). Hughes’s model indicates 
that how participants interpret and react toward a test affects how and what their response to it will be, implying that the quality of a test 
plays an important role in determining washback test effects. However, this model does not clearly delineate washback effects on various 
aspects relating to the teachers/learners and their teaching/learning processes. 

Bailey’s (1996) model of washback, drawn from Hughes’s framework, specifically identifies Hughes’s participants and products. The 
participants include students, teachers, materials writers, and curriculum designers and researchers, whereas products refer to the test 
effects those participants will produce. This model emphasizes the possible interactions of these participants; however, it does not specify 
the possible differences in the impact of tests on individuals. 

Alderson and Wall’s (1993, pp. 120-121) fifteen washback hypotheses address, in greater detail than Hughes’s and Bailey’s models, what 
washback effects might look like on teachers and learners, as stated below: 

• A test will influence the rate and sequence of teaching/learning. 

• A test will influence the degree and depth of teaching/learning. 

• A test will influence attitudes about the content, method, etc., of teaching/learning. 

• Tests that have important consequences will have washback. 

• Tests that do not have important consequences will have no washback. 

• Tests will have washback effects for some learners and some teachers but not for others. 

Furthermore, these hypotheses imply that in addition to a test, there might be other factors that elicit more effects from some learners and 
teachers than they did for others. In addition, not only does the quality of a test affect teaching and learning, but also how it is used in 
different situations affects the strength of washback. However, no explanations have been made for several of their hypotheses regarding 
washback variability. 

Cheng (1997, 2005), Watanabe (2004), and Green (2007) developed the concept of “washback intensity” to delineate possible factors that 
contribute to different forms of “washback variability”. In their viewpoints, participants’ perceptions of test stakes, test quality, and test 
difficulty tend to vary from person to person and therefore lead to differences in the washback effects experienced by individuals. 

Since the theoretical studies discussed above do not involve evidence-based research, researchers such as Alderson & Hamp-Lyons (1996), 
Bailey (1999), Cheng (1997; 2010), and Wall (2005) cite the need for empirical studies to determine the relationship between how 
participants, processes, and products are affected by tests. In other words, washback variability and washback intensity must be empirically 
tested. 

Empirical Washback Studies 

So far the vast majority of foreign language washback studies—such as those by Alderson and Hamp-Lyons (1996), Burrows (2001, 2004), 
Chen (2002), Cheng (2004, 2005), Ferman (2004), Hawkey (2006), Lam (1994), Lewkowicz and Zawadowska-Kittel (2008), Qi (2005), 
Saif (1999, 2006), Valazza, (2008), Watanabe (1996a, 1996b, 2004), Wall and Horak (2006, 2008), and Wu (2008)—have focused on 
teacher perspectives concerning tests. Three types of claims have been made: (1) tests influence both what and how instruction occurs, (2) 
tests impact what is taught but not how, and (3) teacher response to test preparation and changes in tests varies from person to person 
based on both testing (e.g., viewpoint on the quality and content of the test) and mediating factors (e.g., teachers’ educational backgrounds, 
beliefs, concerns for students’ proficiency levels, and psychological factors). 

In contrast to the wealth of teacher washback studies, learner washback research is severely lacking. Wall (2000, p. 502) laments the 
paucity of research and Cheng (2008), Spratt, (2005), and Watanabe (2004) emphasize the need to explore the impact of tests on learners, 
since they are directly affected by these tests. Studies that have focused on learning washback reveal varied and sometimes contradictory 
findings. Three types of student washback claims are documented. 

First, tests influence what is learned, but not how. Cheng’s (1998, 2005), Qi’s (2004, 2005), and Stoneman’s (2006) studies revealed that 
the methods students used to prepare for different exams were similar. Traditional learning strategies and exam preparation methods such 
as reading textbooks, memorizing vocabulary and idioms, going through previous exams, or relying on test prep books were prevalent 
among their research subjects. 



Second, tests have a variety of effects on students, varying according to their different viewpoints of the tests or their different levels of 
language proficiency. Stoneman (2006), Tsagari (2006), and Shohamy et al. (1996) discovered that learners’ perceptions of the stakes and 
status of the test influenced the strength of the test effects: a high-stakes, high-status test promoted learning; students spent more time 
engaging in learning language skills that were covered on the test than they did on lower-stakes or lower-status tests. Similarly, Tsai and 
Tsou (2009) found that negative student opinions on the adoption of standardized exit tests led to a decrease in motivation to learn 
English because their classes were test-oriented, only enhancing their test-taking skills instead of their communicative competence. 

In regard to learner washback from language proficiency, the findings were mixed but appear to contend that students’ levels of English 
proficiency carried some weight in determining the extent of the effort they were likely to make toward a test. According to Stoneman 
(2006), Watanabe (2001), and Chu (2009), low-achieving students tended to be more worried about the test or test requirement than 
high-achieving students, and these low-achieving students did not prepare for the test until the last minute or did not prepare at all. As 
contended by Watanabe, a test of appropriate difficulty for the learner can positively affect his/her motivation to prepare for the test. 
Similarly, Bright and Von Randow (2004) found that low-proficiency students were unwilling or unable to take steps to improve their 
scores because of time concerns, workload, or the stress of coping with new academic environments, although they accepted the accuracy 
of their DELNA (Diagnostic English Language Needs Analysis) scores and knew that language proficiency could affect their academic 
progress. In contrast, Ferman (2004) and Shohamy et al. (1996) found that students with lower abilities, given their belief that cramming 
improves their scores, engage in more intense learning for the test than their counterparts. The higher-ability students were already eager 
to learn, even without the push of the test. 

What is lacking in the aforementioned empirical studies is whether learners’ viewpoints of the tests are related to their language proficiency 
and therefore lead to a variety of learning strategies among individuals. This study therefore aims to investigate learner washback 
variability by exploring the relationship between learners’ viewpoints of the tests, learning strategies, and test performance. 

Research Context 

Since 2003, Taiwan’s Ministry of Education has encouraged universities and colleges of technology to set English thresholds for graduates 
to generate a level of English proficiency sufficient to meet the anticipated needs of both domestic and international job markets. In 
response, the number of four-year technical universities/colleges in Taiwan implementing an English certification exit requirement policy 
for non-English majors increased from 5% in 2003 to 90% in 2012 (Roever & Pan, 2008; Pan, 2010, 2012). To meet the English 
certification exit requirements, students must choose from an array of external English proficiency tests such as the TOEIC®, TOEFL®, 
IETLS™, the General English Proficiency Test (GEPT), and the College Student English Proficiency Test (CSEPT) and reach a certain level 
or score in order to graduate. However, to assist students who did not pass the required English proficiency tests, 

“support/alternative/complementary measures” (translated literally from Mandarin) are provided. These measures include taking remedial 
classes and/or taking an internal in-house test. 

The tertiary institute recruited for this study established its English certification exit requirements in 2008. At the time of this study, 
fourth-year university students were not under the policy, while the remaining three student cohorts had to meet the exit requirements in 
order to graduate. Students are required to show the school their English certification test results by their fourth year at the latest. To assist 
their students with passing any of the English certification tests, in addition to using the regular English reading textbooks adopted before 
the establishment of the exit requirements, students were instructed with either GEPT-related teaching materials or TOEIC-related 
teaching materials. The school library also offers online websites on which students can practice test-related questions that often appear on 
various English certification tests. 

Research Questions 

It is the intent of this study to investigate various learning washback effects on students under different policies (exit vs. non-exit 
requirement policy), different years of study (1st through 3rd year), and different levels of proficiency. For simplicity, “non-exit students” 
refer to those who are not under the English certification exit requirements and “exit students” refer to those who are under such 
requirements. 

1. Did exit students outperform non-exit students on the standardized tests? 

2. Are exit students and non-exit students motivated to take the English proficiency tests required by the exit policy? 

3. Do exit students and non-exit students use different learning activities to study English after class? 



4- Are exit students’ after-class learning activities affected by their years of study and levels of proficiency? 

5. Are exit students’ viewpoints of the tests affected by different levels of proficiency? 

Methods 

Rationale 

Alderson and Wall (1993, p. 117) point out that washback effects exist when “teachers and learners do things they would not necessarily 
otherwise do because of the test.” Similarly, Messick (1996, p. 241) describes washback as “the extent to which the introduction and the use 
of a test influences language and teachers to do things they would not otherwise do that promote or inhibit language learning.” To 
investigate test effects, Wall and Horak (2007) state the importance of baseline studies in the investigation of test impact in that data 
collected at the baseline can be used as a point of comparison to assess if any change caused by the test has taken place. This study, 
therefore, has employed baseline data as a point of comparison to investigate any effects tests have had on students’ motivation, learning 
activities, and test performance. 

Subjects 

The technical university recruited for this study has 11 disciplines, 8 of which belong to the business college, the remainder belonging to the 
science college. In order to make the samples representative, stratified sampling, in which the population was divided into subgroups 
(strata) and random samples were selected from these strata (Castillo, 2009), was utilized. Based on this probability sampling technique, 
three disciplines were chosen from the business school and one was chosen from the science school. In total, 587 students were recruited, 
and a breakdown of these four student cohorts is shown in Table 1. 

As mentioned in the section “RESEARCH CONTEXT,” these students attended the same technical university, which established its English 
certification exit requirements in 2008. At the time of the study, only the fourth-year students had been admitted before the English exit 
requirement regulation had been instituted. As shown in Table 1, 457 students (referred to as “exit students”) were enrolled under the 
English certification exit requirements, and 132 students (referred to as “non-exit students”) were enrolled without such a policy. 

To explore research questions 4 and 5, exit students were grouped into three proficiency levels. Exit students whose scores are below the 
sum of the mean scores minus one standard deviation are considered low-proficiency students (n=68, 14.9%), those whose scores are above 
the sum of the mean scores plus one standard deviation are high-proficiency students (n=77, 16.8%), and those whose scores are between 
the low- and high-proficiency students are the intermediate-level proficiency students (n=3i2, 68.3%). The number of exit students in the 
three proficiency levels follows the curve of standard distribution. 

Table 1. Subject profiles 




Exit students 

(n=457) 

(n=457) 


Non-exit students 

(11=132) 

Gender 

Male 

164 

35-9% 

39 

29-5% 


Female 

293 

64.1% 

93 

70.5% 

Years of study 

First 

180 

39-4% 




Second 

149 

32.6% 

Not applicable 

Third 

128 

28.0% 


Fourth 

Not applicable 


132 100.0% 




Disciplines 

Marketing 

119 

26.0% 

32 

24.2% 


Management 

110 

24.1% 

30 

22.7% 


Leisure Management 

112 

24.5% 

36 

27 - 3 % 


Computer Science 

116 

25.4% 

34 

25.8% 

Test-taking experience 
(school-made test excluded) 

Yes 

122 

26.7% 

46 

34 - 8 % 

(t excluded) 

No 

335 

73 - 3 % 

86 

65.2% 

Pass certification exit requirements? 

Pass GEPT 

112 

24 - 5 % 




Pass TOEIC 

22 

4.8% 

Not applicable 


Pass school-made test 

43 

9.4% 




Not yet pass 

280 

61.3% 




Instruments 

Two student questionnaires, one practice GEPT test, and one practice TOEIC test were utilized to explore the five research questions. 

The GEPT is a four-skill (listening, reading, speaking, and writing) general English proficiency examination commissioned by Taiwan’s 
Ministry of Education in 1999 to encourage the study of English in a communicative way (Wu & Wu, 2007). The TOEIC, which is 
comprised of a Listening Section and a Reading Section, “measures the everyday English skills of people working in an international 
environment” (ETS, 2012). Due to the limited amount of time, the recruited students were required to take the listening and reading 
sections of the practice GEPT test. In addition, the practice TOEIC test was shortened from 200 to too questions, where the first half of the 
questions in each section were used for this study. 

The two student questionnaires were developed based on the results of washback studies conducted by Cheng (2005), Stoneman (2006), 
Green (2007), and Pan (2010). The intent of the first questionnaire was to determine whether there is a difference between exit and non¬ 
exit students in regard to the following two issues. 

• Intention to take external English proficiency tests and motivation for taking such tests 
(in the multiple-choice format) 

• Various types of after-class learning activities ranked by students on a 5-point Likert scale (always, usually, often, seldom, never; the 
Cronbach’s a reliability for the 24 items on a 5-point Likert scale was 0.913.) 

The intent of the second questionnaire was to determine exit students’ viewpoints of the GEPT and TOEIC tests on a 5-point Likert scale 
(from strongly agree to strongly disagree) with regard to their importance in enhancing students’ language skills and motivation to English 
study. The Cronbach’s a reliability for the 28 items on a 5-point Likert scale was 0.928. 

Data Collection and Analysis 

Students were asked to fill in the questionnaires translated into Chinese, and take the GEPT test as the first task and the TOEIC test two 






months later as the second task. The Chinese version of the student questionnaires was delivered by the researcher to student respondents 
to facilitate the process of completing the questionnaire and put the participants at ease (Johnson & Christensen, 2000). Those who 
completed both tasks were given reward vouchers. 

Before the main study commenced, 10 exit students and 5 non-exit students were asked to complete the questionnaires. One student from 
each group was subsequently asked if they were confused by any of the wording of the questionnaire items. Both groups of students agreed 
that each question was clear and comprehensible, so no substantial changes were made. Out of 610, 21 students did not finish the second 
task, resulting in a completion rate of 96.3%. 

Independent exploratory factor analyses using principal component analyses with varimax rotation were conducted on student 
questionnaires for exit and non-exit students. The factorability of the items in the two student questionnaires was examined by the Kaiser - 
Meyer-Olkin measure and Bartlett’s test of sphericity. Factor loadings lower than .3 were deleted and were not counted toward any factors. 

The first questionnaires for both exit and non-exit students 

The factor structure for after-class learning activities used by exit and non-exit students in the first questionnaire (24 items, a=-9i4, see 
appendix 1 for details) was grouped into two major categories: a) types of test-preparation activities and b) types of language skill-building 
activities. The factor structure of the 1st student questionnaires with Cronbach’s a and the number of items was listed in Table 2, as 
follows: 

Types of test-preparation activities 

Exit and non-exit students’ test-preparation activities were grouped into three types: 

(1) AOTP: autonomous out-of-school test preparation, where students practiced self-purchased 4-skill test-preparation materials. 

(2) ASTP: autonomous school website test-preparation, where students practiced reading and listening tests on the online programs 
provided by the school 

(3) ITP: in-class assigned test-preparation, where students practiced the mock tests (e.g., listening, reading, speaking and writing) assigned 
by their instructors. 

Types of language skill-building activities 

The language skill-building activities engaged in by exit and non-exit students were grouped into three types: 

(1) APR: autonomous practice of receptive skills, where students read magazines, newspapers, novels, browsing English-language websites, 
listening to English broadcast programs 

(2) APP: autonomous practice of productive skills, where students employed communicative activities to practice writing and speaking 
skills, such as keeping an English-language diary, talking with foreigners, joining English conversation clubs, and practicing English with 
teachers 

(3) RPRP: required practice of both receptive and productive skills, where students were asked by their instructors to read textbooks, 
memorize vocabulary and idioms, study grammar and sentence patterns, listen to audio versions of textbooks, and practice writing short 
sentences as after-class exercise 

Table 2. First questionnaire: Exit and non-exit students’ after-class test-preparation and language skill-building 
activities 


After-class activities 

Factors 

Number of items used 

Reliability(a) 

Test-related 

AOTP 

Autonomous out-of-school test-preparation 

4 

■939 


ASTP 

Autonomous school online test-preparation 

2 

.892 





ITP 

In-class assigned test -preparation 

4 

.894 

Language skill-building 
(non-test related) 

APR 

Autonomous practice on receptive skills 

5 

.822 


APP 

Autonomous practice on productive skills 

4 

.612 


RPRP 

Required receptive and productive skills practice 

5 

.805 


The second questionnaires for exit students 

The factor structure for exit students’ viewpoints of the GEPT and TOEIC in the second questionnaire (26 items, a=.904, see appendices 2 
and 3 for details) was grouped into three categories, as shown in Table 3. The three categories are as follows: 

(1) ELS (enhancement of language skills): Preparing for GEPT and TOEIC can enhance students’ four language skills 

(2) EEM (enhancement of extrinsic motivation): Passing GEPT and TOEIC can help the students to graduate, continue their education, 
enhance their workplace competitiveness, and therefore is important 

(3) EIM (enhancement of intrinsic motivation): Preparing for GEPT and TOEIC can help the students set learning goals and enhance their 
study time, interest in studying English, and motivation to study English. 


Table 3. Second questionnaire: Exit students’ views of the GEPT and TOEIC tests 


View 

Factors 

Number of Items used 

Reliability(a) 

GEPT/TOEIC 

ELS 

Enhancement of language skills 

4/4 

■ 455 /-756 


EEM 

Enhancement of extrinsic motivation 

4/4 

.702/.743 


EIM 

Enhancement of intrinsic motivation 

5/5 

.873/.868 


Five types of data analyses were adopted to answer the five research questions. First, descriptive analyses of the demographic variables 
were conducted for both questionnaires. Second, exploratory factor analyses were conducted to plot the major constructs of the two 
questionnaires. Third, independent sample t-tests were used to explore the relationships within the two groups of students (exit and non¬ 
exit students) in regard to their test performance and after-class learning activities. Fourth, one-way ANOVA test procedures were 
employed to determine the differences across exit students in different years of study and with different levels of language proficiency. 
Finally, the Chi-square test was employed to analyze the nominal data in regard to exit and non-exit students’ intention to take the English 
proficiency tests. Statistically significant differences with alpha were set at .05 for the data. In addition, effect sizes using Cohen’s d for the 
statistically significant items were calculated to better gauge how the different groups of students varied. Table 4 summarizes the 
participants involved, the instruments adopted, and the procedures of data analysis in this study. 

Table 4. Methodologies 


Participants 

Instruments 

Data Analysis 

457 exit students 

132 non-exit students 

75-item GEPT practice test 
100-item TOEIC practice test 

Independent sample t-test (interval data) 
ANOVA test (interval data) 

457 exit students 

132 non-exit students 

1st student questionnaire 

Descriptive statistics 

Factor analysis 








Independent sample t-test 

ANOVAtest 

Chi-square (nominal data) 

457 exit students 

2nd student questionnaire 

Descriptive statistics 

Factor analysis 

Independent sample t-test 

ANOVAtest 


Results 

The following sections present the results of the five research questions. 

Research Question 1: Did exit students outperform non-exit students on the standardized tests? 

As shown in Table 5, the exit students’ overall exit test scores were better than those of the non-exit students, and the difference between 
their GEPT listening scores was statistically significant, 1(587) = 2.056, p<.os. However, the effect size is small (Cohen d=.263 ), the 
implication of which is that the test-driven policy did not have much effect on the improvement of test scores. 


Table 5. GEPT and TOEIC scores of exit and non-exit students 




Exit students 

(n= 457 ) 

Non-exit students 

(n=l32) 

M 

SD 

M 

SD 

GEPT scores 

Listening* 

86.81 

16.89 

82.39 

16.66 


Reading 

81.57 

17-39 

80.98 

17.47 


Total 

168.38 

30.79 

164.37 

30.64 

TOEIC scores 

Listening 

140.41 

46.86 

137.12 

46.17 


Reading 

89.40 

48.69 

86.02 

48.34 


Total 

229.60 

76.09 

223.10 

79.91 


*t(587)=2.056, p=.040 


Research Question 2: Are exit students and non-exit students motivated to take the English proficiency tests required 

by the exit policy? 

As shown in Table 6, there was a significant difference between the number of exit and non-exit students who intended to take the external 
English certification examinations, 2 (8, N=589) = 153.60, p < .05. As shown in Table 7, the major difference between exit and non-exit 
students in their motivation to take the English proficiency tests is that 60% of exit students took the English certification tests in order to 
meet the exit requirements, while merely 5% of non-exit students had such an intention. 


Table 6. Exit and non-exit students who intend to take English certification exams 



Intend to take exams 

Do not intend to take exams 

Total 

Exit students 

422 (71.8%) 

34 (5.8%) 

457 ( 77 - 6 %) 

Non-exit students 

60 (10.2%) 

72 (12.2%) 

132 (22.4%) 


Table 7. Exit and non-exit students’ motivation to take English certification exams 








Motivation to take English certification exams 

Exit students 

( n = 457 ) 

Non-exit students 
(n=l32) 


n 

% 

n 

% 

1. To pass exit requirements for graduation 

253 

60.0 

3 

5-0 

2. To verify one’s English proficiency level 

183 

43-4 

34 

56.7 

3. To enhance one’s English proficiency level 

271 

64.2 

49 

81.7 

4. To enhance one’s competitiveness in future education 

123 

29.1 

9 

15.0 

5. To enhance one’s competitiveness in future workplace 

319 

75-6 

51 

85.0 

6. To satisfy parents’ expectation 

34 

8.1 

4 

6.7 


Research Question 3: Do exit students and non-exit students use different learning activities to study English after 

class? 

Table 8 shows that English certification exit requirements appear to promote students’ autonomous learning for test preparation by making 
use of in-class test-related materials (ITP) 1(587) = 3.816, p< .05 and school resources (ASTP), t(587) = 5.028, p< .05, and these two 
types of test-preparation were significantly different from those used by their non-exit counterparts. The effect sizes for ITP and ASTP were 
.37 and .513, representing small to medium, respectively. However, there was no statistically significant difference between these two 
groups of students in regard to their language-skill building activities. Traditional language skill building activities (RPRP) such as reading 
textbooks, memorizing vocabulary and idioms, studying grammar and sentence patterns, listening to audio versions of textbooks, and 
practicing sentence patterns were most frequently used by both exit and non-exit students. 


Table 8. Exit and non-exit students’ after-class learning activities 




Exit students 

( n = 457 ) 

Non-exit students 

(n=l32) 



M 

SD 

M 

SD 

Autonomous out-of-school test-preparation 

AOTP 

1.91 

.86 

2.05 

•99 

Autonomous school online test-preparation programs 

ASTP* 

2.27 

•93 

1.82 

.82 

In-class assigned test-preparation 

ITP* 

2.65 

.82 

2-34 

.82 

Autonomous practice on receptive skills 

APR 

2.00 

■63 

2.05 

■63 

Autonomous practice of productive skills 

APP 

l- 5 l 

•49 

1.46 

•50 

Required receptive and productive skills practice 

RPRP 

2.66 

•63 

2.68 

.69 


See appendix 1 for details. Test-related activities: AOTP, ASTP, ITP; language skill-building activities: APR, APP, RPRP 
ASTP: 1(587) = 5.028, p=.ooo; ITP: 1(587) = 3.816, p=.000 

Research Question 4; Are exit students’ after-class learning activities affected by their years of study and levels of 

proficiency? 

As shown in Table 9, first-year exit students had a significantly higher frequency of doing school online test-preparation practice than their 
second- and third-year counterparts. Second-year exit students decreased the frequency of doing test-preparation practice but increased 
the frequency of doing non-test-related type of practice on receptive skills. The frequency of third-year exit students practicing self- 
purchased test-preparation materials and receptive skills was markedly higher than that of first-year exit students. The implication of this 
finding is that as the time to submit their test scores approaches, the higher the frequency of both autonomous practice on test preparation 







and receptive skills. 

Table 9. After-class learning activities from first- to third-year exit students 

l st -year 2 n< ^-year 3 rt ®-year 

Exits (n=l8o) Exits (11 = 149) Exits (n=l28) 




M 

SD 

M 

SD 

M 

SD 

Autonomous out-of-school test-preparation 

AOTP* 

1.69 

.76 

1.98 

■94 

2.14 

■83 

Autonomous school online test-preparation 

ASTP* 

2.50 

•99 

t -93 

■79 

2-34 

.89 

Required in-class test-preparation 

ITP 

2.71 

.81 

2.61 

.87 

2.63 

.78 

Autonomous practice on receptive skills 

APR* 

1.90 

.60 

2.07 

■65 

2.06 

•63 

Autonomous practice on productive skills 

APP 

1.49 

•43 

1.50 

■52 

1-55 

■54 

Required receptive and productive skills practice 

RPRP 

2.71 

.67 

2.63 

■65 

2.63 

■53 


See appendix 1 for details. Test-related activities: AOTP, ASTP, ITP; language skill-building activities: APR, APP, RPR 
AOTP: F(2,455)= 11.578, p=.ooo; ASTP :F( 2 , 455 )= 16.519, p=.ooo ; APR:F( 2 , 455 )= 3 - 578 , p=. 02 g 

As shown in Table 10, proficiency levels did not bring about significant differences in the frequency of doing online test-preparation 
practice on the school website. However, proficiency levels brought about significant differences in the frequency of doing autonomous and 
required practice on self-purchased test preparation materials and language-skill building activities. In other words, higher-proficiency 
students utilized a variety of resources for test preparation (self-purchased materials, and in-class test-related materials) and employed 
various types of practice of receptive and productive skills as compared to low-proficiency exit students. 

Table 10. After-class learning activities of exit students with different levels of language proficiency 




High 
Exits (n- 

=68) 

Intermediate 
Exits (n=3l2) 

Low 
Exits (n 

= 77 ) 



M 

SD 

M 

SD 

M 

SD 

Autonomous out-of-school test-preparation 

AOTP* 

2.05 

.76 

1.94 

■83 

1.66 

1.03 

Autonomous school test-preparation 

ASTP 

2.41 

1.04 

2.22 

■91 

2-37 

•92 

In-class test-preparation 

ITP* 

2-75 

■93 

2.69 

.80 

2.41 

■77 

Autonomous practice on receptive skills 

APR* 

2.28 

•71 

1.99 

.60 

1.78 

•58 

Autonomous practice on productive skills 

APP* 

1-73 

.60 

1.48 

.46 

1-45 

■49 

Required receptive and productive skills practice 

RPRP* 

2.80 

■59 

2.67 

.62 

2.50 

.67 


See appendix 1 for details. Test-related activities: AOTP, ASTP, ITP; language skill-building activities: APR, APP, RPRP 

AOTP: F(2,455) = 4 - 312 , p=.oi 4 ; ITP: F(2,455) =4-26o, p=.Oi 5 ; APR: F(2,455) =11.714, P=-OOo; APP: F(2,455 ) = 7 - 749 , P=-OOo; RPRP: 

F(2,455) = 4 - 392 , p=.Oi3 

Research Question 5: Are exit students’ viewpoints of the tests affected by different levels of proficiency? 

As shown in Table 11, the exit students had significantly different viewpoints of the English certification exit tests, the GEPT and TOEIC. 
When asked (1) whether passing the GEPT and the TOEIC can help them to graduate, continue their education, and enhance their 
workplace competitiveness, and (2) whether preparing for GEPT and TOEIC can help them to set learning goals and enhance their study 




time, interest in studying English, and motivation to study English, exit students held a more favorable view of the TOEIC than of the 
GEPT. They assumed that the TOEIC was more beneficial in regard to enhancing their extrinsic motivation (EEM), 1(457) = 9.06, p<. 05, 
and intrinsic motivation (EIM), 1(457) = 6.15, p<. 05. 

Table 11. Exit students’ views of the GEPT and TOEIC 




M 

SD 

t 

p 

-value 

Exit students 

( n = 457 > 

GEPT ELS 

3-89 

■757 

.22 

.82 

TOEIC ELS 

3-88 

•559 




GEPT EEM 

4.00 

.603 

-9.06 

.00* 


TOEIC EEM 

4-25 

.518 




GEPT EIM 

3-58 

■759 

-6.15 

.00* 


TOEIC EIM 

3-71 

■730 




See appendices 2 and 3 for details. ELS: enhancement of language skills; EEM: enhancement of extrinsic motivation; EIM: enhancement of 
intrinsic motivation 

As shown in Table 12, the intermediate- and high-proficiency exit students had more favorable views of the tests than low-proficiency 
students. However, higher-proficiency-level exit students appeared to believe that preparing for these exit tests are more likely to enhance 
their language skill proficiency and improve their intrinsic motivation, compared to lower-proficiency-level exit students. 

Table 12. Views of different levels of exit students 



High Exits 
(n= 68 ) 

Intermediate Exits 
(n=3l2) 

Low Exits 

(n =77 ) 

M 

SD 

M 

SD 

M 

SD 

*GEPT ELS 

3-85 

•55 

3-95 

.82 

3-67 

■56 

*TOEIC ELS 

3-94 

•52 

3-94 

■55 

3.62 

■55 

GEPT EEM 

3-91 

.68 

4.02 

■58 

4.02 

■59 

TOEIC EEM 

4-25 

•37 

4.27 

■52 

4.14 

■58 

*GEPT EIM 

3-65 

•87 

3-64 

■71 

3-32 

.76 

*TOEIC EIM 

3-85 

•71 

3-77 

■71 

3-35 

■72 


See appendices 2 and 3 for details. ELS: enhancement of language skills; EEM: enhancement of extrinsic motivation; EIM: enhancement of 
intrinsic motivation 

(ELS), GEPT F(2,455) =4-39L P=-Oi 3 ; TOEIC F(2,455 ) =10.331, P=-000; (EIM), GEPT F(2,455) =5-717, P=-004; TOEIC F(2,455) =11.765, 

p=.000. 

Discussion 

Test impact on test performance, motivation to take tests, and test-related and language skill-building activities 

The major aim of this study was to investigate how exit tests influence learners’ motivation, learning activities, and test performance. The 








English certification exit requirements have brought about several effects. First, the exit students’ overall exit test scores were better than 
the non-exit students,’ and the difference between their GEPT listening scores was statistically significant. Second, the exit requirements 
for English have motivated students to take external English proficiency tests. Third, English certification exit requirements appear to 
promote students’ autonomous learning for test preparation by making use of in-class test-related materials and school resources, and the 
use of these two types of test-preparation by exit students was significantly different from their use by non-exit counterparts. Fourth, such 
a test-driven policy did not change students’ language skill-building activities to any significant degree because there was no significant 
difference between exit and non-exit students in regard to the frequency of their non-test-related practice on receptive and productive 
skills. Since the type of school online test-preparation primarily involved practice of receptive skills such as listening and reading, this may 
at least partially account for exit students scoring significantly higher on the GEPT listening test than non-exit students. However, 
increased scores resulting from test-preparation activities may not necessarily represent a corresponding increase in language abilities 
because such test-taking skills may merely help test-takers who do not possess the target skills to guess the right answers, therefore leading 
to the increase in their scores (Messick, 1982). 

The GEPT and TOEIC, according to their developers, are designed to test learners’ language abilities in their daily lives and in the business 
world respectively, so both receptive and productive skills are assessed. However, there is no significant difference between exit and non¬ 
exit students in terms of their frequency of adopting traditional methods and communicatively-oriented methods to study English. These 
traditional methods such as reading textbooks, memorizing vocabulary and idioms, studying grammar and sentence patterns, listening to 
audio versions of textbooks, and writing short sentences were actually reported by these two groups of students to be the most frequently 
used. Communicatively-oriented methods such as keeping an English-language diary, talking with foreigners, joining English conversation 
clubs, and practicing English with teachers were used the least often by these two groups of students (see Table 8). In view of this, it is a 
difficult task to change students’ study habits by means of a test-driven policy, as found in the studies by Cheng (1998), Stoneman (2006), 
Chu (2009), and Xie (2013). For example, Xie (2013) found, through the results of 873 student questionnaires, that rehearsing test-taking 
skills was the most frequently used test preparation practice, but “learning strategies focused on the development of language skills via 
extensive and functional uses of English language were used scarcely” (p. 210). This result may be partially due to the fact that English, in 
these researchers’ contexts, is learned as a foreign language, not a second language. English is studied as a “school subject,” and learners do 
not have many opportunities to utilize what they have prepared for the tests in their daily lives, especially in their school lives. In other 
words, the English certification exit requirements appear to have promoted “assessment of learning,” but it is a challenging task to 
determine how to utilize such a test-driven policy to promote “assessment for learning,” where learners can use what they have prepared 
for the tests in the real world (Cheng et al., 2010). 

Test impact on learner washback variability 

Another objective of this study was to test Alderson’s and Wall’s hypotheses regarding learner washback variability: 1) A test will have 
washback for some learners but not for others, 2) A test will influence the rate and sequence of learning, and 3) A test will influence the 
degree and depth of learning. Based on the findings drawn from exit students’ views on different tests, their years of study, and their levels 
of English proficiency, this study appeared to confirm these three hypotheses. 

Because the TOEIC is an international test (whereas the GEPT is a local test), the exit students probably held a more favorable view of the 
TOEIC than they did of the GEPT, assuming that the TOEIC was more beneficial in regard to enhancing their extrinsic motivation and 
intrinsic motivation. According to student questionnaire 2 (see Table 3 and appendices 2 & 3 for details), extrinsic motivation refers to the 
stakes of the test, where passing the test can help one to graduate, to continue his or her education, or to enhance his or her job readiness, 
and therefore it is an important test. Intrinsic motivation refers to preparation for the test being able to help one to set his or her learning 
goals or increase the amount of time one studies or one’s motivation to study English. This finding is consistent with findings from Stone’s 
study (2006), in which students were more motivated to prepare for the higher-status exam (the International English Language Testing 
System) than the lower-status test (the Graduating Students Language Proficiency Assessment). However, the students in both studies 
preferred traditional methods of preparing for the test such as “doing practice exercises using past papers and/or test preparation books” 

(p. 398). Therefore, for the students participating in these two studies, tests do not influence their strategies for learning English or 
preparing for the test. 

Intermediate- and high-proficiency exit students had more favorable views of the exit tests than their low-proficiency counterparts did. 
Significant differences were found in exit student views of whether or not these exit tests can improve their language skill proficiency and 
enhance their intrinsic motivation. It is interesting to note that there were no differences of opinion in regard to whether these exit tests 
can enhance extrinsic motivation among these three proficiency levels of exit students. In other words, the three proficiency levels of 
students all contended that these two exit tests were important for their graduation, job applications, and further education. The test they 
took made no difference to these low-proficiency exit students other than its test stakes, due to the fact that the test requirement was a 




source of extrinsic motivation that resulted in increased test preparation in the hope of passing the test. This probably explains why lower- 
proficiency exit students primarily engaged themselves in certain types of test-specific practice such as after-class learning activities, 
whereas higher-proficiency exit students adopted various test-related and language skill-building activities (see Table to). In addition, 
high- and intermediate-proficiency exit students are also aware of which test can enhance their intrinsic motivation more and which test 
can enhance their language skills more. This finding may suggest that when teachers and administrators are choosing exit tests for low- 
proficiency students, the stakes of the test are what these people need to take into account due to the fact that the low-proficiency students 
will choose learning activities to practice based upon whether they regard the test as important for their ability to graduate, to seek 
employment, or to further their education. 

The types of learning activities adopted by the first-through-third-year exit students appear to confirm Alderson and Wall's hypothesis 
regarding how a test influences the rate and sequence of learning. Alderson and Wall wanted to determine how quickly and in what order 
tests would influence learning. First-year exit students conducted school online test-preparation practice to a significantly higher degree. 
The frequency to which second-year exit students did test-preparation practice decreased, but they spent an increased amount of time on 
non-test-related practice of receptive skills. Compared to their first-year counterparts, third-year exit students had a significantly increased 
frequency of doing various types of test-preparation and non-test-related types of practice of receptive skills (see Table 9). These findings 
suggest that, as time progresses, exit requirements could influence the frequency and order of how students conduct both test-related and 
non-test-related practice. The results for third-year exit students confirmed the results of Shohamy et al.’s study (1996), in which the 
number of test-related activities in which participants engaged increased as the date of the test approached. But the results for the second- 
year exit students showed a decrease in frequency of test preparation. This decreased amount of test preparation frequency for the second- 
year students is probably because no test-preparation classes were offered for them, and there was still some time before they were 
required to submit their test scores to meet the exit requirements. 

The different proficiency levels of exit students also confirmed Alderson and Wall’s hypothesis regarding how a test influences the degree 
and depth of learning. According to Alderson and Wall, degree and depth refer to the quality and quantity of learning influenced by tests. 
Although proficiency levels did not indicate significant differences in the frequency of online test-preparation practice, English certification 
exit requirements appear to affect both quality and quantity of higher-proficiency students' learning activities (such as a variety of 
resources for both test-related and non-test-related activities). Furthermore, low-proficiency students did not spend as much time on 
certain types of test preparation and language-skill building activities as intermediate- and high-proficiency students did, but the English 
certification exit requirements encouraged them to make efforts to utilize school resources for test preparation, as the intermediate- and 
high-proficiency students did. This could be considered to be a positive test effect for low-proficiency students, because at least they spent 
some of their time on test preparation. 

Implications 

Taking all of the results from the two practice tests and the two student questionnaires into consideration, there are several pedagogical 
implications and suggestions. 

First, given the insignificant difference in test scores on reading skills between exit and non-exit students, other than test-specific practice, 
this finding indicates that test preparation did not make much difference in reading skills for the students. Due to the fact that the GEPT 
was established to encourage the study of English in a communicative way (Wu, 2007) and the TOEIC was designed to measure the 
everyday English skills of people in the workplace (ETS, 2007), students should be encouraged to engage in a greater number of 
communicatively-oriented activities to practice their reading skills, such as reading magazines, newspapers, and novels, and browsing 
English-language websites. 

Second, intermediate- and high-proficiency students appear to have benefited from the English certification exit requirements more than 
low-proficiency students because the latter had a lower frequency of doing both autonomous test-specific and language skill-building 
activities than the former. In addition, low-proficiency exit students had less favorable viewpoints of the two exit tests than their higher- 
proficiency counterparts. Considering this result, various measures of exit requirements could be employed for students with different 
levels of proficiency. It is suggested that, in addition to results from a formal standardized exit test, multiple measures (e.g., formative 
assessments) that match students’ proficiency levels and needs must be developed as part of the criteria for exit requirements. This would 
be much more likely to foster positive effects (Berry & Lewkowicz, 2000; Chu, 2009; Saif, 1999, 2006; Tsai & Tsou, 2009). The formative 
assessments could include different types of information such as teacher-developed and portfolio assessments to improve the quality and 
validity of the decisions made based upon student performance (Berry & Lewkowicz, 2000; Henderson-Montero, Julian & Yen, 2003). By 
introducing multiple measures as part of exit requirements, students will also be encouraged to develop a broader range of language skills 



than can be assessed on a one-off language test (Pan & Newfields, 2012). 

Third, exit students used school resources more often than non-exit students to study English after class. In light of this finding, a variety 
of school online educational resources covering practice of the four language skills should be provided from which students could choose so 
that students will have more opportunities of doing both test-related and communicatively-oriented practice. 

Conclusion 

Through an analysis of student questionnaires and test scores, this study has provided us with a better understanding of learning washback 
in regard to how tests influence student performance, learning strategies, and motivation. In addition, it highlights the differential effects of 
exit requirements for different groups of learners depending on their years of study, proficiency levels, and perceptions of tests. 

This study has three limitations that should serve as suggestions for further research. First, its focus is limited to a policy of mandating EFL 
proficiency in a Taiwanese tertiary context among non-English majors. Whether or not the patterns in this study will generalize to 
respondents with different educational backgrounds or fields other than English is an open question. Future studies should explore how 
exit exam policies appear to impact test stakeholders in different learning environments. Second, this study might have been able to 
investigate substantial and apparent changes brought about by the tests if it had been conducted a few years after the exit tertiary institute 
had implemented the policy for a longer period. A longitudinal study would surely offer a clearer picture of the long-term effects of this exit 
exam policy. Subsequent research should include classroom observation and interview data in order to generate a clearer picture of how 
washback patterns on learners are presented. 
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Appendix 1 

1st student questionnaire: types of exit and non-exit students’ after-class test-preparation and language skill-building activities 

Directions: In this section, students were asked how often they did the following activities after class and had to tick one of the given boxes 
for 1) never, 2) seldom, 3) sometimes, 4) usually, and 5) always. 


After-class 

Activities 

Factors 

Items 

Test-related 

AOTP 

Autonomous out-of-school test-preparation 

Practice self-purchased test preparations for reading skills 


Practice self-purchased test preparations for listening skills 



Practice self-purchased test preparations for writing skills 

Practice self-purchased test preparations for oral skills 


ASTP 

Autonomous school online test-preparation 

Practice reading tests on the online programs provided by the school 
Practice listening tests on the online programs provided by the school 


ITP 

In-class assigned test -preparation 

Practice the mock listening tests assigned by the instructor 


Practice the mock reading tests assigned by the instructor 



Practice the mock speaking tests assigned by the instructor 



Practice the mock writing tests assigned by the instructor 


Language skill-building APR Read English magazines 

(non-test related) Autonomous practice on receptive skills 

Read English newspapers 


Read English novels 


Browse English-language websites 


Listen to English broadcast programs 

APP Read English magazines 

Autonomous practice on productive skills Read English newspapers 

Read English novels 

Browse English-language websites 

RPRP Read textbooks required by the instructor 

Required receptive and productive skills 

Memorize vocabulary and idioms required by the instructor 
Study Grammar and sentence patterns required by the instructor 
Listen to audio versions of textbooks required by the instructor 
Practice writing short sentences required by the instructor 


Appendix 2 

2nd student questionnaire: types of exit students’ views of the GEPT test 

Directions: In this section, students were asked how much they agreed with the following items and had to tick one of the given boxes for 
1) strongly disagree, 2) disagree, 3) not sure, 4) agree, and 5) strongly agree. 


Types of tests 

Factors 

Items 

GEPT 

ELS 

Enhancement of language skills 

Preparing for the GEPT test can enhance my vocabulary and grammar 


Preparing for the GEPT test can enhance my listening and reading skills 



Preparing for the GEPT test can enhance my writing and speaking skills 

Preparing for the GEPT test can help me to learn the communicative ability 


EEM 

Enhancement of extrinsic motivation 

Passing the GEPT test can help me to graduate 

Passing the GEPT test can help me to continue my education 

Passing the GEPT test can help me to enhance my workplace competitiveness 
Passing the GEPT test is every essential for me 


EIM 

Enhancement of intrinsic motivation 

Preparing the GEPT test can help to set my learning goals 


Preparing for the GEPT test can help to enhance my study time 



Preparing for the GEPT test can help to enhance my interest in studying English 



Preparing the GEPT test can help to enhance my motivation to study English 
Preparing the GEPT test can help me to be willing to study harder for English 


Appendix 3 

2nd student questionnaire: types of exit students’ views of the TOEIC test 

Directions: In this section, students were asked how much they agreed with the following items and had to tick one of the given boxes for 
1) strongly disagree, 2) disagree, 3) not sure, 4) agree, and 5) strongly agree. 

Types of tests Factors 


Items 



TOEIC 

ELS 

Enhancement of language skills 

Preparing for the TOEIC test can enhance my vocabulary and grammar 


Preparing for the TOEIC test can enhance my listening and reading skills 



Preparing for the TOEIC test can enhance my writing and speaking skills 

Preparing for the TOEIC test can help me to learn the communicative ability 


EEM 

Enhancement of extrinsic motivation 

Passing the TOEIC test can help me to graduate 

Passing the TOEIC test can help me to continue my education 

Passing the TOEIC test can help me to enhance my workplace competitiveness 
Passing the TOEIC test is every essential for me 


EIM 

Enhancement of intrinsic motivation 

Preparing the TOEIC test can help to set my learning goals 


Preparing for the TOEIC test can help to enhance my study time 



Preparing for the TOEIC test can help to enhance my interest in studying English 



Preparing the TOEIC test can help to enhance my motivation to study English 
Preparing the TOEIC test can help me to be willing to study harder for English 
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